In conventional interrupt architectures, the interrupt delivery mechanism can create a high latency event. Interrupt pins are usually shared between multiple devices, leading to an ambiguity as to the originating source of an interrupt appearing on the shared interrupt pin. This ambiguity makes it necessary for the individual device drivers to read their associated devices over a Peripheral Component Interconnect (PCI) bus to determine if the interrupt was generated by the device, and if so, what caused the interrupt.
FIG. 1 illustrates a conventional server or computer 100 with interrupt handling capability. In FIG. 1, the server 100 includes a host 102, which includes a processor 138 and firmware 140. The firmware 140 contains instructions executable by the processor 138 for implementing an Operating System (OS) 104, one or more drivers 106, and an interrupt controller 108. The interrupt controller 108 may be connected to a PCI bus 110, a PCI-eXtended (PCI-X) bus 112 or a PCI Express bus 114 (collectively “a PCI-based bus”) through a host bridge 128.
Coupled to the PCI-based bus are n+1 Host Bus Adapters (HBAs) 116, each HBA for enabling communications between devices coupled to the PCI-based bus and an external network 118 such as a Storage Area Network (SAN) over a link 120 such as a Fibre Channel (FC) link. Note that there is one driver 106 in host 102 for every HBA 116 coupled to the PCI-based bus. The HBAs 116 are configured by firmware 160 to provide an interface between the host 102 and the network 118.
The HBAs 116 may support multiple channels and one or more protocols (e.g. FC, Serial Attached Small computer system interconnect (SAS), Serial Advanced Technology Attachment (SATA) or the like). Each channel on the HBA 116 may utilize a SERializer/DESerializer (SERDES) 142 and a protocol core ENGine (ENG) 144 coupled to the SERDES 142 for communicating over the network 118. A Bus Interface Unit (BIU) 146 couples one or more ENGs 144 to Random Access Memory (RAM) 148 and to one or more processors 150 and provides an interface to an internal bus 152. Also connected to the internal bus 152 is an Address Translation Unit (ATU) or host interface 154 for interfacing between the internal bus 152 and the PCI-based bus.
The HBAs 116 process attention conditions and generate interrupts using firmware 160 and interrupt logic 156 located in the BIU 146. Each HBA 116 may experience a number of different conditions known as attention conditions, each of which may create a different type of interrupt. Interrupts are HBA-specific notifications sent to the HBA's associated device driver 106 to indicate that particular events have occurred. In the example of FIG. 1, the interrupt logic 156 in HBA0 116 includes an attention register 122 containing m+1 (e.g. 32) attention bits (e.g. ATT0-ATTm), each representing a different type of attention condition, and a conditions register 126. When an attention condition is experienced from within an HBA 116, the appropriate attention bit for that attention condition is asserted in the attention register 122, and the conditions register 126 is updated with information indicating what caused the attention condition. An interrupt must thereafter be passed up to the host 102 so that the interrupt can be processed by the driver 106 associated with that HBA 116.
In servers 100 utilizing a PCI bus 110 or a PCI-X bus 112, a single shared interrupt bit in the PCI or PCI-X bus is asserted whenever an attention bit is asserted in the attention register 122 in any of the HBAs 116, and this bit is passed up to the interrupt controller 108. However, because the interrupt bit is shared, the asserted interrupt bit provides no indication to the interrupt controller 108 as to which HBA 116 generated the interrupt. In servers 100 utilizing a PCI Express bus 114 (a serial bus), a serial interrupt message is generated and passed up to the interrupt controller 108. Again, the interrupt message also provides no indication to the interrupt controller as to which HBA 116 generated the interrupt.
Because the interrupt bit or message provides no indication as to which HBA 116 generated the interrupt, the interrupt controller 108 forwards the interrupt bit or message to all of the drivers 106 in the host 102. Each driver 106 must then separately query its associated HBA 116 and read the attention register 122 and the conditions register 126 to determine if that HBA generated the interrupt, and if so, what caused the interrupt. Of course, only one of the drivers 106 will determine that its associated HBA 116 generated the interrupt, but the additional traffic generated by the queries from the other drivers leads to the consumption of costly processor cycles and increased traffic on the PCI-based bus. After the driver 106 whose associated HBA 116 generated the interrupt is identified, and the asserted attention bits that generated the interrupt are identified, the driver can the process the interrupt.
While the current interrupt bit or message is being processed by the appropriate driver, it is possible for the HBA 116 to experience a new attention condition and assert a corresponding attention bit in the attention register 122 that would normally cause the same interrupt bit or message to be generated again. However, because the interrupt bit or message currently being processed by the driver will not contain any information about this newly asserted attention bit 122, the new attention condition will not be processed by the driver. When the attention bits that caused the current interrupt bit or message to be generated are subsequently cleared when processing of the current interrupt bit or message is completed, the newly asserted attention bit will remain asserted. However, in some implementations, the newly asserted attention bit will not cause a new interrupt to be sent, and the new attention condition will effectively have been lost.
To avoid losing attention conditions in the manner described above, an attention enable register 124 may be employed in each HBA 116. There is one attention enable bit (e.g. one of ATTENB0-ATTENBm) for every attention bit in the attention register 122. In systems utilizing attention enable registers 124, after the driver 106 whose associated HBA 116 generated the interrupt bit or message is identified, and the asserted attention bits that caused the interrupt are identified, the driver commences processing of the interrupt bit or message by sending a write command to the attention enable register 124 to deassert (i.e. disable) the attention enable bits associated with all of attention bits in the attention register 122 that could have caused the interrupt message to be generated. Once an attention enable bit is deasserted, if the HBA experiences an attention condition associated with that attention enable bit and asserts the appropriate bit in the attention register 112, no interrupt corresponding to that attention enable bit will be immediately generated. When the processing of the current interrupt has been completed, the driver 106 processing the interrupt acknowledges the interrupt by sending a write command to the HBA 116 to clear down (deassert) all previously identified attention bits in the attention register 122. Note, however, that the newly asserted attention bit will not be deasserted. The driver 106 also sends another write command to the attention enable register 124 to assert (enable) the previously deasserted (disabled) attention enable bits in the attention enable register. When the attention enable bits are asserted again, the newly asserted attention bit will then generate a new interrupt bit or message. The net result is that the attention condition that caused the newly asserted attention bit will be processed as a result of the new interrupt, and will not be lost. However, these additional write commands also lead to increased traffic on the PCI-based bus and consumption of costly processor cycles.
MSI, as described in the PCI Local Bus Specification Revision 3.0, incorporated herein by reference, is an optional feature for PCI devices and required for PCI-X devices. MSI-eXtended (MSI-X, also defined in PCI Local Bus Specification Revision 3.0) is an optional extension to MSI. In either MSI or MSI-X, an MSI engine 160 within each HBA 116 composes and sends an in-band memory write message containing information identifying the HBA that generated the interrupt, and also what caused the interrupt. The write message is received by the interrupt controller 108 and stored in host memory in a location configured by the OS 104 and the drivers 106. The interrupt controller 108 is able to read the write message and forward it only to the driver 106 associated with the HBA 116 that generated the interrupt. This write message therefore eliminates the need for each driver 106 to query its associated HBA 116 to determine if that HBA generated the interrupt, and to determine which attention bits were asserted, thereby reducing bus traffic and improving the processing efficiency of the server 100. However, use of the MSI and MSI-X feature does not eliminate the multiple write commands necessary to deassert the attention enable bits in the attention enable register 124, clear down the attention register 122, and re-assert the attention enable bits in the attention enable register in order to avoid losing interrupts.
Therefore, there is a need to eliminate the need for drivers to send write commands to the HBAs during the processing of interrupts while still ensuring that no interrupts are lost.